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1  INTRODUCTION 

This  demonstration  is  designed  to  illustrate  the  discrimination  performance  at  a  challenging  live- 
site  of  advanced  electromagnetic  induction  (EMI)  modeling  approaches,  that  are  based  on  the 
normalized  surface  magnetic  source  NSMS  model.  The  model  is  an  extension  of  the  simple 
dipole  model,  and  provides  better  accuracy  and  discrimination  ability.  The  approach  combines  1. 
the  NSMS  for  modeling  targets  EMI  responses;  2.  the  differential  evolution  (DE)  algorithm  for 
nonlinear  optimization  to  locate  the  target,  and  3.  Statistical  classification  approach  for 
classifying  targets  as  UXO  and  non-UXO.  The  study  used  cued  data  sets  collected  at  San  Luis 
Obispo,  Calaifornia  using  three  next-generation  EMI  sensors,  the  Geometries  MetalMapper 
(MM),  the  Time-domain  Electro-Magnetic  Towed  Array  Detection  System  (TEMTADS) 
developed  by  the  NRL  and  G&G  Sciences,  and  Berkley  UXO  Discriminator  (BUD)  developed  at 
the  Lawrence  Berkeley  National  Laboratory  (LBNL).  The  site  was  contaminated  with  60  mm,  81 
mm,  2.36  inch  and  4.2  inch  munitions.  During  this  study,  first  targets  extrinsic  (location  and 
depth)  and  intrinsic  (the  total  NSMS,  which  depends  on  its  size,  shape  and  material  properties) 
parameters  were  estimated  from  the  data.  Then,  the  inverted  intrinsic  parameters  were  used  to 
classify  the  targets,  and  finally,  sensor-specific  dig-lists  were  generated  for  each  EMI  instrument 
and  submitted  to  the  Institute  of  Defense  Analyses  (IDA)  for  independent  scoring. 

1.1  Background 

The  Environmental  Security  Technology  Certification  Program  (ESTCP)  has  recently  launched  a 
series  of  live-site  UXO  blind  tests  taking  place  in  increasingly  challenging  and  complex 
environments  [1-4].  The  first  classification  study  was  conducted  in  2007  at  the  UXO  live-site  at 
the  former  Camp  Sibert  in  Alabama  using  two  commercially  available  first-generation  EMI 
sensors  (the  EM61-MK2  and  the  EM-63,  both  from  Geonics)  [1].  At  that  site,  the  discrimination 
test  was  relatively  simple:  one  had  to  discriminate  large  intact  4.2"  mortars  from  smaller  range 
scrap,  shrapnel  and  cultural  debris,  and  the  anomalies  were  very  well  separated. 

The  second  ESTCP  discrimination  study  took  place  in  2009  at  the  live-UXO  site  at  Camp  San 
Luis  Obispo  (SLO)  in  California  and  featured  a  more  challenging  topography  and  a  wider  mix  of 
targets  of  interest  (TOI)  [4].  Magnetometers  and  first-generation  EMI  sensors  (again  the  Geonics 
EM61-MK2)  were  deployed  on  the  site  and  used  in  survey  mode  for  a  first  screening. 
Afterwards,  two  advanced  EMI  sensing  systems — the  Berkeley  UXO  Discriminator  (BUD)  and 
the  Naval  Research  Laboratory’s  TEMTADS  array — were  used  to  perform  cued  interrogation  of 
a  number  of  the  anomalies  detected.  A  third  advanced  system,  the  Geometries  MetalMapper,  was 
used  in  both  survey  and  cued  modes  for  anomaly  identification  and  classification.  Among  the 
munitions  buried  at  SLO  were  60-mm,  81-mm,  and  4.2"  mortars  and  2.36”  rockets. 


1.2  Brief  site  history 

Please  refer  to  the  ESTCP  Live  Site  Demonstration  Plan  [3]. 
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1.3  Objective  of  the  demonstration 

The  advanced  EMI  models  we  present  here  (NSMS  and  JD)  were  developed  under  SERDP 
Project  MM-1572  [5]  and  tested  against  TEMTADS  data  sets  collected  at  the  Aberdeen  Proving 
Ground  (APG)  test  site  in  Maryland  [6-10].  The  present  test  of  discrimination  performance 
considers  data  taken  at  SLO.  This  live  site  was  contaminated  with  60-mm,  81-mm,  and  4.2" 
mortars  and  2.36"  rockets;  and  three  additional  types  of  munitions  were  discovered  during  the 
course  of  the  demonstration. 

Overall,  the  principal  objective  of  this  demonstration  was  to  demonstrate  the  models’ 
classification  performance  for  live-site  UXO  problems.  The  specific  technical  objectives  were  to: 

1.  Demonstrate  the  classification  accuracy  of  the  NSMS  model  and  its  applicability  to  live-site 
UXO  discrimination  problems  in  terms  of  the  signal  to  noise  ratio  and  number  of  targets. 

2.  Illustrate  and  document  the  robustness  of  the  data  inversion  and  discrimination  models. 

3.  Invert  targets’  intrinsic  parameters  and  identify  robust  classification  features. 

4.  Indentify  all  seeded  and  native  UXO. 

5.  Indentify  sources  of  uncertainty  in  the  classification  process  and  include  them  in  a  dig/no¬ 
dig  decision  process. 

6.  Understand  and  document  the  applicability  and  limitations  of  the  advanced  EMI 
discrimination  technologies  in  the  context  of  project  objectives,  number  of  targets,  site 
characteristics,  and  suspected  ordnance  contamination. 
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2  TECHNOLOGY 

The  advanced  EMI  models  and  statistical  signal  processing  approaches  have  been  developed  and 
tested  under  this  SERDP  Project  MM- 1572  [5].  Namely  the  methods  were  tested  against  the 
TEMTADS  data  sets  from  the  APG  test-site.  The  methods  were  able  to  detect  and  identify 
buried  UXO  ranging  in  caliber  from  25  mm  up  to  155  mm.  The  technique  was  seen  to  be 
reasonably  fast,  accurate,  and  clutter-tolerant,  and  provided  excellent  classification  in  single¬ 
target  scenarios  when  combined  with  multi-axis/transmitter/receiver  sensors  like  TEMTADS 
[11].  To  further  evaluate  the  performance  of  this  technology,  in  this  work  advanced  EMI  data 
inversion  and  classification  studies  are  conducted  for  SLO,  CA  live-site. 

We  start  our  technology  description  with  the  overview  of  the  NSMS  (section  2.1),  which  we  use 
to  represent  the  signatures  and  extract  the  properties  of  subsurface  targets  in  an  efficient  manner. 
In  Section  2.2  we  outline  the  Gaussian  mixture  model  for  target  classification.  In  Section  2.3  we 
discuss  the  geometries  and  sensing  modalities  of  the  advanced  EMI  sensors,  as  well  as  the 
procedures  we  have  in  place  to  model  the  way  the  sensors  establish  primary  fields  and  measure 
subsurface  responses.  We  first  describe  the  MetalMapper,  and  then  continue  with  TEMTADS 
and  BUD.  Finally,  in  Section  2.4  we  present  the  SLO  data  sets  classification  results  obtained 
using  the  combined  NSMS  approach  and  advanced  classification  procedures. 

2.1  The  Normalized  Surface  Magnetic  Source  model 

The  most  frequently  used  method  for  representing  the  EMI  response  of  a  metallic  target  in  both 
frequency  and  time  domains  approximates  the  whole  object  with  a  set  of  orthogonal  co-located 
point  dipoles  that  fire  up  in  response  to  the  primary  field;  the  induced  dipole  moment  is  related  to 
the  primary  field  through  a  symmetric  polarizability  tensor.  The  use  of  this  dipole  approximation 
is  motivated  by  its  speed  and  simplicity;  this  simplicity,  however,  rests  on  assumptions  that  often 
become  problematic  and  limit  the  model’s  usefulness.  One  such  assumption  is  that  the  buried 
target  of  interest  is  either  far  enough  from  the  transmitter  loop,  or  small  enough,  that  the  primary 
field  is  essentially  uniform  throughout  its  extent.  Usually,  complex  targets  composed  of  different 
materials  and  different  sections  that  contribute  appreciably  to  the  response — and,  in  the  case  of 
UXO,  containing  such  complicating  features  as  fins  and  rings — simply  cannot  be  modeled 
accurately  with  a  single  point  dipole.  Such  cases  require  more  advanced  methods  that  will 
capture  the  underlying  physics  correctly.  One  such  technique  is  the  NSMS  model. 

The  NSMS  method  [12-15]  can  be  considered  as  a  generalized  surface  dipole  model,  and  indeed 
reduces  to  the  point  dipole  model  in  a  special  limiting  case.  The  NSMS  approach  models  an 
object’s  response  to  the  primary  field  of  a  sensor  by  distributing  a  set  of  equivalent  elementary 
magnetic  sources — normally  oriented  dipoles  in  this  case — over  an  auxiliary  surface  that 
surrounds  it.  Such  a  surface  distribution  can  be  hypothetically  generated  by  spreading  positive 
magnetic  charge  over  the  outer  side  of  the  equivalent  surface  (usually  a  prolate  spheroid)  and  an 
identical  distribution  of  opposite  sign  on  its  inner  side  [16],  resulting  in  a  double  layer  of 
magnetic  charge  separated  by  an  infinitesimal  distance.  This  double  layer  introduces  the  proper 
discontinuities  in  the  tangential  components  of  the  magnetic  flux  density  vector  B  but  does  not 
affect  the  transition  of  its  normal  component,  which  must  always  be  continuous  given  the  lack  of 
free  magnetic  charges  in  nature.  The  resulting  magnetic-moment  distribution  radiates  a  field  that 
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by  construction  satisfies  the  governing  EMI  equations  and  can  thus  account  for  the  secondary 
field  outside  the  object.  The  particulars  of  location  and  orientation  are  divided  out  by 
normalizing  the  dipole  density  at  every  point  with  the  component  of  the  primary  magnetic  field 
normal  to  the  surface.  The  resulting  surface  amplitude  Q  of  the  NSMS  distribution  is  a  property 
of  the  object,  and  its  integral  Q  over  the  surface  constitutes  a  sort  of  global  magnetic 
polarizability  that  is  independent  of  the  computational  constructs — primary  field,  surrounding 
surface,  object  location  and  orientation,  etc. — introduced  for  its  determination.  The  surface 
amplitude  can  be  determined  directly  for  library-matching  purposes  by  minimizing  the  difference 
between  measured  and  modeled  data  for  a  known  combination  of  object  and  sensor  at  a  given 
relative  location  and  orientation. 


The  NSMS  model  is  based  upon  the  assumption  that  the  entire  scatterer  can  be  replaced  with  a 
very  thin  auxiliary  surface  shell.  The  primary  magnetic  field  strikes  the  shell  and  induces  on  it  a 
surface  magnetization.  The  normal  component  of  the  magnetic  field  flux  is  always  continuous 
across  a  boundary  (no  free  magnetic  charges  exist  in  the  universe)  between  two  media,  the  total 
scattered  magnetic  field  can  be  represented  with  surface  magnetic  sources  as 


H 


:(r)  =  d)  G(r,  r ')  •  m(,y  ')ds'  =  d)  G(r,r') 

s  s 


M(s')  •  H,"’(s') 


ds' 


(1) 


where 


G(r,r') 


1 

4  ttR3 


3RR  = 

R2 


R=r-r'. 


(2) 


Thus  the  EMI  response  of  a  permeable  and  conducting  metallic  object  can  be  represented  using 
a  surface  density  on  magnetic  dipoles  m(s'),  that  is  proportional  to  a  normalized  surface 

distribution  M(s')  through 


m(s')  =  M(s  ’)  •  Hpr(s')  ,  (3) 

the  integral  of  M(.v  j  over  the  surface  provides  the  total  NSMS,  Mmm(  =<£  M(s’)<A',  contains  all 

the  information  about  an  object  that  could  be  of  need  in  the  UXO  discrimination  problem, 
incorporating  the  effects  of  heterogeneity,  interaction  with  other  objects,  and  near-  and  far-field 
effects. 


2.2  Gaussian  mixture  models 

2.2.1  Model-based  supervised  clustering 

Targets  of  interest  (TOI)  with  similar  features  (i.e.,  dipole  polarizabilities  or  total  NSMS)  are 
likely  to  show  similar  power-law/exponential  time  decay  patterns  under  various  conditions,  and 
as  a  result  these  patterns  form  clusters  when  plotted  in  a  convenient  and  pertinent  space.  It  is 
possible  to  identify  an  unknown  target  by  comparing  its  time-decay  parameters  to  those  of  a  set 
of  previously  characterized,  previously  clustered  objects  and  assigning  it  to  the  category  where 
its  profile  fits  best.  Such  “supervised”  clustering  allows  the  use  of  additional  information  from 
the  training  data  as  prior  knowledge;  on  the  other  hand,  it  uses  only  the  training  set  to  estimate 
the  parameters  and  completely  ignores  the  blind  data,  which  is  potentially  quite  useful.  The  test 
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data  set,  moreover,  is  usually  much  larger  than  the  training  sample,  implying  that  the  unused 
information  may  be  substantially  richer  than  what  is  contained  in  the  training  sample. 

Let  us  assume  that  there  are  K  clusters  and  that  each  cluster  is  mathematically  described  by  a 
parametric  continuous  or  discrete  distribution  function  (usually  a  Gaussian,  as  in  this  case).  The 
classification  parameters  (extracted  for  example  by  fitting  the  total  NSMS  or  ONVMS  with  a 
Pasion-Oldenburg  time-decay  law)  can  then  be  arranged  in  an  n  x  m  matrix  denoted  by 
Y  =  [Yi,  Y2,  ...,  Y m\,  where  Y,-,  i  =  1,  2,  ...,  n,  is  a  vector,  n  is  the  number  of  anomalies,  and  m  is 
the  number  of  parameters.  Each  Y,  can  be  considered  to  follow  an  m-dimensional  mixture  of 
normal  distributions  expressed  as 


)  =  £»,/, (Y, 'ft.®,).  (4) 

k=l 

where  wk  is  the  mixing  weight  of  cluster  k  (defined  as  the  proportion  of  anomalies  that  belong 
to  it),  Zf=1w,  =  1 ,  and 


m  1  IV * )  =  /  1  expf  - \ c Y  - H* )'cr"1(Y.  - \xk )  I  (5) 

W2*)  ^  2  > 

is  the  probability  density  of  the  k- th  normal  distribution  with  the  mx  1  mean  vector  pA  and  the 
mxm  variance-covariance  matrix  ak .  The  parameters  ,  a,  ,  and  wk  are  estimated  by  the 
maximum  likelihood  (ML)  criterion  using  the  Expectation  Maximization  (EM)  algorithm. 

This  simple  and  intuitive  supervised  method  usually  performs  well  if  the  number  of  TOI  within 
each  known  cluster  in  the  training  sample  is  sufficiently  large  to  ensure  high  accuracy  of  the 
estimates  of  p;  and  ok  .  For  small  training  samples  these  estimates  are  subject  to  large  errors — 
in  some  cases,  if  the  number  of  targets  within  a  cluster  is  smaller  than  the  number  of  parameters, 
the  estimated  variance-covariance  matrices  may  not  even  be  positive  definite.  Furthermore, 
much  information  from  the  test  dataset  has  not  been  fully  utilized.  The  test  dataset  is  usually 
much  larger  than  the  training  sample,  implying  that  the  unutilized  information  may  be 
substantially  more  than  that  contained  in  the  training  sample.  To  overcome  this  problem  we  use 
unsupervised  clustering  and  derive  pA  and  a  k  from  blind-test  data  using  an  iterative  EM 
algorithm. 

2.2.2  Unsupervised  classification  using  the  multivariate  normal  mixture  approach 

Mixture  distribution  is  perhaps  the  only  model-based  approach  among  existing  methods  of 
clusterization  and  pattern  recognition.  Its  attractive  features  are  that  (a)  it  is  not  necessary  to 
specify  what  class  each  observation  belongs  to  (i.e.,  the  classification  is  “unsupervised”)  and  that 
(b)  the  method  estimates  the  membership  probability  which  results  in  confusion  matrix  n . 

Let  x  ,...,x  be  m-dimensional  feature  vectors  that  we  want  to  split  into  K  classes.  It  is  assumed 
that  each  x.  belongs  to  one  of  K  classes  that  are  described  by  densities  (pfx),...,(pK{x)  .If  nk>  0 
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denotes  the  probability  of  x  belonging  to  class  k ,  then  the  mixture  density  is  the  linear 
combination 

K 

^(x)  =  ^/r^(x),  (6) 

k= 1 

where 


2 X=1- 


In  the  case  of  a  normal  distribution  we  have 


K 

JjrkN(x;iik,£lk) 


k= 1 


(7) 


(8) 


for  the  mixture,  where  the  mean  p;  and  the  covariance  matrix  flk  are  different  for  the  different 
clusters  and  subject  to  estimation  along  with  probabilities  nk .  Since  the  density  of  AAx^^Q, )  is 


<pk(x;nk,nk)  =  (2xrm'2nk-  exp 


-|(x-l**WI(x-H*) 


(9) 


the  method  of  maximum  likelihood  prescribes  the  maximization  of  a  nonlinear  function: 


MnA  .*■*)= w"/2  r  ~  exp 


k= 1 


max 

,7Tk 


(10) 


This  task  is  not  easy  because  the  function  is  unbounded  (it  may  go  to  +oo),  and  this  creates 
numerical  obstacles.  The  expectation-maximization  (EM)  algorithm  is  used  to  determine 
nk,\ik,C\  as  follows: 


1.  Let  the  initial  estimates  of  7tk,\xk,C\  be  given  (for  example,  we  can  take  rck=\ IK, 
o,  =  Sx,  and  from  K ). 


2.  Compute  the  elements  of  the  confusion  matrix  ji  as  the  probability  that  a  given 
observation  x.  belongs  to  class  k  : 


nik  -  T - 

^;(x;p;,0;) 

j-[ 

3.  Adjust  the  probabilities  accordingly: 


n 


k 


n 


7t 


ik 


(11) 


(12) 
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4.  Recompute  the  means 


Ha 


y v  x. 

ik  i 

i= 1 _ 


nn, 

k 


and  covariance  matrices 


where: 


_2u„Sa-+S,; 

2u„  +  ’ 


(13) 


(14) 


Sa=Z^a(x;-Ha)(x,-Ha)'-  (15) 

1=1 

5.  If  the  values  are  different  from  the  previous  iteration  return  to  Step  1  and  continue 
until  reaching  convergence. 


2.3  Modeling  of  advanced  EMI  systems 

A  wide  range  of  different  electromagnetic  induction  sensing  technologies,  with  novel 
waveforms,  multi-axis  transmitters,  and  scalar/vector  receivers  have  been  recently  developed 
under  SERDP-ESTCP  programs.  These  advanced  EMI  sensors — including  the  MetalMapper,  the 
TEMTADS  array,  and  the  Berkeley  UXO  discriminator  (BUD)  provide  measurements  that 
feature  a  combination  of  high  spatial  diversity,  different  viewpoints,  and  a  very  wide  dynamic 
range  and  which  do  full  justice  to  the  vector  character  of  the  electromagnetic  field. 

2.3.1  MetalMapper 

The  MetalMapper  (MM)  is  an  advanced  EMI  system  for  UXO  detection  and  discrimination 
developed  primarily  by  G&G  Sciences  and  commercialized  by  Geometries.  The  system  has  three 
mutually  orthogonal  transmitter  rectangular  loops.  It  is  able  to  illuminate  a  target  with  primary 
fields  from  three  independent  directions  from  a  single  spatial  field  point.  The  1  m  x  1  m  Z 
transmitter  loop  is  located  at  ground  level.  The  Y  transmitter  loop,  also  1  m  x  1  m,  is  centered  56 
cm  above  the  Z  loop,  as  is  the  0.98  m  x  0.98  m  X  transmitter  (Figure  1).  The  targets  are 
illuminated  from  different  directions  depending  on  the  geometry  between  a  particular 
transmitting  loop  and  the  target.  The  system  has  seven  10-cm-side  receiver  cubes  placed  at  seven 
unique  spatial  points  on  the  plane  of  the  Z  transmitter  loop.  The  receivers  measure  the  vector 
dB/dt  at  each  of  the  seven  points,  thus  providing  63  independent  readings  of  the  transient 
secondary  magnetic  field  for  each  instrument  location.  The  positions  of  the  receiver  cubes’ 
centers  with  respect  to  the  Z  transmitter  loop  (whose  center  we  consider  as  the  local  origin  of 
coordinates  for  the  system)  are  given  in  Table  1. 
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Figure  1:  The  MetalMapper  during  SLO  site  deployment  (left)  and  its  schematic  diagram  (right). 
Table  1:  MetalMapper  receiver  locations  with  respect  to  the  center  of  the  Z  transmitter  loop. 


Rx# 

X  [cm] 

Y  [cm] 

Z  [cm] 

0 

39 

39 

5 

1 

-26 

26 

5 

2 

13 

13 

5 

3 

0 

0 

5 

4 

-13 

-13 

5 

5 

26 

-26 

5 

6 

-39 

-39 

5 

The  MM  transmitters  are  modeled  as  infinitely  thin  rectangular  wires.  The  primary  magnetic 
induction  produced  at  any  observation  point  r  by  the  T-th  loop  is  determined  simply  from  the 
Biot-Savart  law, 


B 


A^/r[Alr.xRr.] 


Ri 


T  =  1,2,3, 


(16) 


where,  R7  /  -r-r(.,,  r^.is  the  location  of  the  i-th  current  element,  and  a£Ti  is  the  tangential 
length  vector  for  the  i-th  subsection  of  the  Tth  loop.  In  what  follows,  and  unless  we  note 
otherwise,  we  divide  each  transmitter  coil  into  N.,.  =  40  subsections  whenever  we  calculate  the 

7  Tx 

primary  magnetic  induction  using  Eq.  (16). 
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Transmitter  c 


Receiver  cubes 


m 


Figure  2.  The  MetalMapper  geometry.  The  observation  point  r  is  defined  with  respect  to  the 
global  Cartesian  coordinate  system  XYZO;  r3'.  is  the  location  of  the  i-th  current  element  on  (in 

this  case)  the  T=  3  transmitter,  which  carries  a  current  /3  in  the  direction  £  . 

The  MM  receiver  assembly  consists  of  seven  cube  sensors.  Each  of  these  measures  along  three 
orthogonal  directions  the  induced  voltages  that,  from  Faraday’s  law,  correspond  to  the  negative 
of  the  time  derivative  of  the  secondary  magnetic  flux  through  the  area  spanned  by  the  different 
coils.  The  induced  voltage  in  the  R-th  sensor  along  the  a- th  direction,  where  R- 0,  ,6  and 

a=  z,y,x,  is  computed  using 


(17) 


where  sR  is  the  area  of  the  relevant  coil  (all  of  which  are  10  cm  x  10  cm  squares  in 
MetalMapper)  and  h  is  the  unit  vector  perpendicular  to  it,  A s“R  and  r“R  are  respectively  the  i  - 
th  sub-area  and  vector  location  point  on  sR ,  B  (0  =  A0H(r“)  is  the  magnetic  induction 
(proportional  to  the  magnetic  field  H  (r",) )  produced  at  r“R  by  a  source  placed  at  r  .  Within  the 
NSMS  model,  H.(r.“R)  is  calculated  using  equation  (1)  in  section  2.1.  In  what  follows  we  always 
divide  into  Na  =4  sub-areas. 

R  Rx 
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Figure  3:  Photo  of  the  TEMTADS  in  deployment  at  APG  site  (left)  and  a  schematic  diagram  of 
its  Tx/Rx  sensors  (right). 

2.3.2  TEMTADS 

The  NRL  time-domain  EMI  sensor  array  TEMTADS  is  a  next-generation  system  designed  for 
subsurface  target  discrimination.  The  sensor  consists  of  25  transmit/receive  pairs,  each  composed 
of  a  35-cm  square  transmitter  loop  surrounding  a  25-cm  square  receiver  loop,  arranged  in  a 
rectangular  5x5  grid  with  40-cm  neighbor-to-neighbor  separation  [11]  (Figure  3).  The  sensor 
activates  the  transmitter  loops  in  sequence,  one  at  a  time,  and  for  each  transmitter  all  receivers 
receive,  measuring  the  complete  transient  response  over  a  wide  dynamic  range  of  time  going 
approximately  from  100  microseconds  (ps)  to  25  milliseconds  (ms)  and  distributed  in  123  time 
gates.  The  sensor  thus  provides  625  spatial  data  points  at  each  location,  with  unprecedented 
positional  accuracy. 

In  modeling  for  TEMTADS,  the  transmitter  loops  are  idealized  as  infinitesimally  thin 
35  cm  x  35  cm  square  loops.  The  primary  field  produced  at  any  observation  point  by  a  given 
transmitter  loop  is  determined  from  equation  (16).  We  use  NTx  =  20  for  TEMTADS  unless  we 
note  otherwise.  The  TEMATDS  measured  signal  is  modeled  using  equation  (17),  assuming 
a  =  z  throughout  and  receiver  sizes  of  25  cm  x  25  cm  and  dividing  each  receiver  into  NRx  =  9 
sub-areas. 
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Figure  4:  Schematic  diagram  of  the  BUD  system. 

2.3.3  BUD 

The  Berkeley  UXO  discriminator  (BUD)  is  an  advanced  standalone  time-domain  system 
developed  at  the  Lawrence  Berkeley  National  to  detect  and  discriminate  UXO  in  the  20-mm  to 
155 -mm  size  range,  and  consists  of  three  orthogonal  coil  transmitters.  The  horizontal  Z-coils  are 
vertically  separated  by  26"  and  have  a  39"  x  39"  footprint.  The  Y-  and  X- vertical  coils  are 
mounted  on  the  diagonals  between  the  Z-coils  (see  Figure  4):  the  X-coils  are  45.5"  x  23.5"  while 
the  y-coils  are  45.5"  x  22.5"  in  size,  and  both  are  separated  by  6".  The  BUD  illuminates  targets 
in  three  independent  directions,  which  induce  eddy  currents  in  all  three  modes.  BUD  has  eight 
pairs  of  differenced  receiver  coils  placed  horizontally  along  the  two  diagonals  of  the  upper  and 
lower  planes  of  the  Z-transmitter  loops.  The  pairs  are  located  on  symmetry  lines  trough  the 
center  and  are  wired  in  opposition  so  as  to  cancel  the  primary  magnetic  field  during  transmission. 
Figure  5  shows  the  BUD  system  in  operation. 

The  BUD  transmitter  loops  were  modeled  as  idealized  infinitely  thin  square  loops.  The  primary 
fields  produced  at  any  observation  point  by  the  transmitters  are  determined  using  a  suitable 
modification  of  equation  (16),  again  with  NTx  =  40.  The  BUD  measured  signals  are  modeled 
using  equation  (17)  as 


' '  Rx 

v*  =  -I 


dB/fr;.*  ~ro) 
8t 


■As„.  +  Ia  —  As,. 


dt 


A Sj.R  —  As,.  »z 


(18) 


where  r  and  r!  are  the  locations  of  the  Rx  and  Rx'  receivers,  given  in  Table  2.  For  the  case  of 
BUD  we  divide  the  receivers  into  N  D  =9  sub-areas. 

Rx 
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Figure  5:  The  BUD  system  in  operation. 

Table  2:  BUD  receiver  locations  with  respect  to  the  origin. 


Rx# 

X  [cm] 

Y  [cm] 

Z 

[cm] 

Rx'# 

A '[cm] 

7' [cm] 

Z'[cm] 

1 

35.48 

35.48 

0 

r 

-35.48 

-35.48 

66 

2 

-35.48 

35.48 

0 

2’ 

35.48 

-35.48 

66 

3 

-35.48 

-35.48 

0 

3’ 

35.48 

-35.48 

66 

4 

35.48 

-35.48 

0 

4’ 

-35.48 

35.48 

66 

5 

19.29 

19.29 

0 

5’ 

-19.29 

-19.29 

66 

6 

-19.29 

19.29 

0 

6’ 

19.29 

-19.29 

66 

7 

-19.29 

-19.29 

0 

7’ 

19.29 

19.29 

66 

8 

19.29 

-19.29 

0 

8’ 

-19.29 

19.29 

66 
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2.4  SLO  data  sets  classification  results 

Our  group  conducted  SLO  classification  studies  using  the  advanced  EMI  models.  Namely,  the 
combined  NSMS-DE  algorithm  was  used  for  data  inversion  and  classification  parameter 
extractions,  and  library  matching  and  Gaussian  mixture  models  were  used  for  classification. 
Each  anomaly  was  treated  as  a  single  well  separated  target,  i.e.  no  multi-target  scenarios  were 
considered.  The  target  response  was  approximated  with  set  of  magnetic  dipoles  distributed  on  a 
spherical  surface  of  radius  5  cm.  This  sphere  is  divided  into  17  subsurfaces,  each  of  which  is 
assumed  to  contain  a  magnetic -dipole  distribution  of  constant  density.  Once  the  location  of  the 
sphere’s  center  is  determined  then  the  magnitude  of  each  responding  source  is  obtained  and  the 
total  NSMS  is  calculated.  The  SERDP  Program  Office  provided  us  with  training  data  sets  for  the 
analysis  of  the  algorithms  testing  performance. 

2.4.1  SLO  TEMTADS  Classification 

The  SERDP  office  provided  us  the  188  TEMTADS  calibration  data,  that  we  used  to  build  a 
library  of  the  expected  total  NSMS  values  for  TOI.  This  library  was  then  used  on  the  rest  1282 
test  cells  for  classifying  anomalies  as  TOI-and  non-TOI-s.  We  used  all  115  TEMTADS  time 
channels  that  span,  in  approximately  logarithmic  fashion,  a  lapse  of  time  between  100  ps  and  24 
ms.  The  inversion  code  assumed  that  the  TEMTADS  instrument  was  always  placed  30  cm  above 
the  ground. 

For  each  data  set  we  run  the  combined  NSMS-DE  to  estimate  object  extrinsic  (locations  and 
depth)  and  intrinsic  (total  NSMS)  parameters.  The  inverted  total  NSMS  curves  for  SLO 
TEMTADS  calibration  (green  lines)  and  blind  data  sets  (red  lines)  are  depicted  in  Figure  6  and 
Figure  7  for  partial  2.36"  rockets,  4.2”  mortars,  81-mm  projectiles,  2.36"  rockets,  and  60-mm 
mortars.  The  results  indicate  that  the  inverted  and  calibration  total  NSMS  time  decay  curves  are 
similar  and  are  good  discriminators.  Also,  as  the  size  of  the  TOI  decreases  the  inverted  total 
NSMS  time  decay  curves  show  a  larger  spread,  making  them  harder  to  discriminate. 
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Figure  6:  Inverted  total  NSMS  time  decay  profiles  for  the  2.36"  partial  rocket.  The  green  lines 
depict  calibration  data  and  the  red  lines  correspond  to  blind  SLO  TEMTADS  data  sets. 


10 

Time  [m  Sec] 


Time  |m  Sec)  Time  tm  Sec) 

Figure  7:  Inverted  total  NSMS  time  decay  profiles  for  4.2”  mortars  (top  left),  81 -mm  projectiles 
(top  right),  2.36"  rockets  (bottom  left),  and  60-mm  mortars  (bottom  right)  in  the  SLO 
TEMTADS  test.  The  green  lines  depict  calibration  data  and  the  red  lines  correspond  to  blind  data 
sets. 
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Figure  8:  Result  of  the  supervised  clustering  classification  for  the  SLO-TEMTADS  anomalies 
using  the  logarithms  of  Af^Oj)  and  MrJt;)  /  .  The  supervised  clustering  has  been  trained 

with  calibration  data.  The  red  markers  correspond  to  clutters  and  the  white  ones  to  TOI. 

Once  the  total  NSMS  were  determined,  the  values  of  logio(  Maa(t1)/  Maa(tm) )  versus 
logio( were  extracted  for  each  anomaly,  and  semi- supervised  Gaussian  mixture  model 

was  used  to  classify  the  targets  as  TOI-and  non-TOI.  These  values  are  plotted  in  Figure  8  for  all 
TEMTADS  data  sets  as  red  and  white  dots.  In  addition,  Figure  8  shows  distribution  of  the 
Gaussian  mixture  classification  function  (color  map  distributed  between  blue  (zero  level)  and  red 
(maximum  value,  one)  colors).  We  see  that  the  inverted  parameters  are  well  clustered,  and,  those 
of  TOI  are  noticeably  distinct  from  all  others,  suggesting  that  this  two-dimensional  feature  space 
should  provide  good  classification. 

We  combined  the  Gaussian  mixture  model  and  library  matching  classification  results  and  built  a 
dig  list.  The  dig  list  was  submitted  to  the  IDA  for  scoring.  The  scored  results  in  ROC  curve  form 
are  depicted  for  all  TOI  and  individual  TOI-s  in  Figure  9  and  Figure  10. 
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DartmouthSky  TEMTADS  AdvancedModels  TestSet  TOI 


Figure  9:  ROC  curve  for  SLO  TEMTADS  test  data. 
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Figure  10:  ROC  for  SLO  TEMTADS  data  for  individual  TOI. 
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The  results  show  that  our  classification  approach  was  able  to  correctly  identify  most  TOI.  It  was 
able  to  classify  large  4.2"  and  81  mm  mortars  and  well-separated  2.36  "  rockets,  60  mm 
mortars,  which  produced  high  signal  to  noise  ratios.  It  had  difficulties  to  tag  correctly  two  2.36" 
rockets  and  five  60-mm  mortars,  which  were  mostly  placed  with  other  targets  or  had  small 
signal-to-noise  ratios. 

2.4.2  SLO  MM  Classification 

The  SERDP  office  provided  us  2492  SLO  cued  MM  data  sets.  Each  data  set  was  inverted  using 
the  combined  NSMS-DE  algorithm  for  single  objects,  to  extract  the  extrinsic  and  intrinsic 
parameters  of  the  targets.  The  extracted  intrinsic  parameters  (NSMS)  were  used  to  classify 
targets  as  UXO  and  non-UXO  targets  and  a  prioritized  dig  list  was  created.  The  dig  list  was  then 
submitted  to  the  SERDP  office  for  independent  scoring,  which  was  carried  out  by  personnel 
from  the  Institute  for  Defense  Analyses  (IDA).  Our  discrimination  results  are  summarized  in 
Figure  11  and  Figure  12.  Our  classification  technique  was  able  to  correctly  identify  all  big  UXO, 
(the  2.36",  81-mm  and  4.2"  projectiles)  for  MetalMapper  data.  The  algorithm  missclassified  only 
one  seeded  60-mm  mortar,  MM  anomaly#  1285.  The  missed  TOI  was  buried  at  35  cm  depth,  and 
the  inverted  total  NSMS  for  the  anomaly  are  significantly  different  compared  to  the  total  NSMS 
for  a  library  60  mm  mortar,  Figure  13.  Based  on  these  differences,  the  anomaly  #1285  was 
ranked  as  non-TOI  target. 


DartmouthSky  MetalMapper  AdvancedModels  TestSet  TOI 


Figure  11:  ROC  curve  for  SLO  MetalMapper  test  data. 
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Figure  12:  ROC  curve  for  SLO  MetalMapper  data  sets:  individual  TOI. 
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Figure  13:  The  total  NSMS  for  MM  anomaly  #1285  (solid  lines)  and  a  library  60  mm  mortar 
(dashed  lines). 
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Figure  14:  ROC  curves  for  SLO  BUD  discrimination  studies. 

2.4.3  SLO  BUD  data  inversion  and  classification  studies 

The  SERDP  office  also  provided  SLO  BUD  data  sets  (total  539  anomalies)  for  assessing  our 
technology  for  the  BUD  system.  We  adapted  NSMS_DE  approach  to  the  BUD  data  sets  and 
applied  it  to  first  the  SLO  training  data  sets  (in  total  69)  and  then  to  SLO  live  site  BUD  data.  We 
extracted  targets  intrinsic  (total  NSMS)  and  extrinsic  parameters  for  each  of  the  anomalies.  The 
discrimination  features  (size  and  shape  information)  were  extracted  from  the  total  NSMS  time 
decay  curve  and  anomalies  were  classified  using  the  library  matching  and  Gaussian  mixture 
model.  The  inverted  targets  were  ranked  as  TOI  and  non-TOI  items.  The  ROC  curve  for  the  SLO 
BUD  data  sets  that  was  ranked  independently  in  our  group,  is  shown  in  Figure  14.  The  studies 
showed  that  only  2.36  inch  rockets  were  misclassified.  In  both  cases  the  signals  from  the  missed 
targets  were  contaminated  with  signals  from  nearby  non-UXO. 
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3  PERFORMANCE  OBJECTIVES 

The  performance  objectives  of  this  ESTCP  live  site  discrimination  study  were:  to  achieve  high 
probability  of  discrimination  of  UXO  from  among  a  wide  spread  of  clutter;  to  process  all  data 
sets;  to  minimize  the  number  of  data  that  could  not  be  analyzed  or  decided  upon;  to  minimize  the 
number  of  false  positives;  and  to  identify  all  UXO  with  high  confidence.  The  performance 
objectives  are  summarized  in  Table  3. 


Table  3:  Performance  objectives 


Performance 

Objective 

Metric 

Data  Required 

Success  Criteria 

Maximize  correct 

classification  of 

munitions 

Number  of  targets-of- 
interest  retained 

•  Prioritized  anomaly 
lists 

•  Scoring  reports  from 
Institute  for  Defense 
Analysis  (IDA) 

Approach  correctly 

classifies  all  targets-of- 
interest 

Maximize  correct 

classification  of  non¬ 
munitions 

Number  of  false  alarms 
eliminated 

•  Prioritized  anomaly 
lists 

•  Scoring  reports  from 
IDA 

Reduction  of  false 
alarms  by  >  75%  while 
retaining  all  targets  of 
interest 

Specification  of  no-dig 
threshold 

Probability  of  correct 
classification  and 

number  of  false  alarms 
at  demonstrator 

operating  point 

•  Demonstrator  - 
specified  threshold 

•  Scoring  reports  from 
IDA 

Threshold  specified  by 
the  demonstrator  to 
achieve  criteria  above 

Minimize  number  of 
anomalies  that  cannot 
be  analyzed 

Number  of  anomalies 
that  must  be  classified 
as  “Unable  to  Analyze” 

•  Demonstrator  target 
parameters 

Reliable  target 

parameters  can  be 
estimated  for  >  90%  of 
anomalies  on  each 
sensor’s  detection  list. 

Correct  estimation  of 
target  parameters 

Accuracy  of  estimated 
target  parameters 

•  Demonstrator  target 
parameters 

•  Results  of  intrusive 
investigation 

Total  NSMS  ±  10% 
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3.1  Objective:  maximize  correct  classification  of  munitions 

An  effective  technology  for  discrimination  of  munitions  should  maximize  the  number  of  targets 
of  interest  (TOI)  it  can  classify  as  such  (thus  distinguishing  them  from  non-TOI)  with  high 
confidence. 

3.1.1  Metric 

Identify  all  seeded  and  native  TOI  with  high  confidence  using  advanced  EMI  discrimination 
technologies.  Our  estimates,  for  "High  confidence”,  were  based  on  using  the  extracted  total 
NSMS  as  input  to  statistical  classification  algorithms  and  expert  judgment.  Every  anomaly  that 
was  close  to  a  TOI  cluster  in  feature  space  (Figure  8)  and  had  /  >  10  7 ,  where/is  the  probability 
density  function  of  equation  (5),  was  considered  to  be  a  possible  TOI;  the  expert  then  inspected 
the  corresponding  TNSMS  curve  for  symmetry  (manifested  by  equal  secondary  and  tertiary 
TNSMS)  and  signal-to-noise  ratio. 

3.1.2  Data  requirements 

We  analyzed  data  from  three  instruments,  the  5x5  TEMTADS  array,  the  MetalMapper  and 
BUD.  For  each  sensor  we  used  supplied  training  data  sets.  The  training  data  were  used  to 
validate  the  models  for  each  specific  sensor.  We  generated  dig-lists  that  were  scored  by  the  IDA 
for  TEMTADS  and  MM,  and  by  ourselves  for  BUD  test  anomalies  ,  respectively.  Our  studies 
showed  that  the  supplied  training  data  provided  a  little  or  no  critical  information  for  targets 
classification;  Therefore,  we  suggested  to  use  a  custom  training  data  set  for  future  classification 
studies. 

3.1.3  Success  criteria  evaluation  and  results 

The  objective  was  considered  to  be  met  if  all  seeded  and  native  UXO  items  could  be  identified 
below  an  analyst-specified  no-dig  threshold. 

3.1.4  Results 

The  objective  was  not  met.  Our  algorithms  classified  most  TOI  correctly,  however  missed  seven, 
one  and  two  TOI  for  TEMTADS,  MM  and  BUD  sensors,  respectively.  Retrospective  analysis 
showed  that  missed  targets  were  in  multi-target  environment(see  Figure  14),  or  buried  deep  and 
had  small  SNR. 

3.2  Objective:  maximize  correct  classification  of  non-munitions 

The  technology  aims  to  minimize  the  number  of  false  negatives,  i.e.,  maximize  the  correct 
classification  of  non-TOI. 
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3.2.1  Metric 

We  compared  the  number  of  non-TOI  targets  that  can  be  left  in  ground  with  high  confidence 
using  the  advanced  EMI  discrimination  technology  to  the  total  number  of  false  targets  that  would 
be  present  if  the  technology  were  absent. 

3.2.2  Data  requirements 

This  objective  required  prioritized  anomaly  lists,  which  our  team  generated  independently  for 
each  sensor,  and  for  its  evaluation  we  needed  scoring  reports  from  the  IDA  or  ground  truth  from 
the  SERDP  office  for  retrospective  studies. 

3.2.3  Success  criteria  evaluation  and  results 

The  objective  was  considered  to  have  been  met  if  the  method  eliminated  at  least  75%  of  targets 
that  did  not  correspond  to  TOI  in  the  discrimination  step. 

3.2.4  Results 

This  objective  was  not  met.  In  this  study,  only  single  object  NSMS-DE  algorithm  was  used  for 
targets  parameters  inversion  and  classification.  The  classification  algorithm  was  unable  to 
classify  all  TOI-s  correctly,  particularly  in  case  of  multi  targets  and  when  targets  were  buried 
deep  i.e.  small  SNR.  To  overcome  these  problems  in  the  future,  we  developed  the  ortho- 
normalized  volume  magnetic  source  (ONVMS)  and  the  joint  diagonalization  (JD)  techniques  for 
modeling  multi  targets  response  and  estimating  number  of  potential  targets.  The  ONVMS  and  JD 
techniques  were  successfully  demonstrated  during  camp  Butner,  NC  [17]  and  Camp  Beale,  CA 
[17]  ESTCP  classification  studies. 

3.3  Objective:  specify  a  no-dig  threshold 

This  project  aims  to  provide  a  high-confidence  classification  approach  for  UXO-site  managers. 
A  critical  quantity  for  minimizing  the  residual  risk  of  UXO  and  providing  regulators  with 
acceptable  confidence  is  a  specific  no-dig  threshold. 

3.3.1  Metric 

We  compared  an  analyst’s  no-dig  threshold  point  to  the  point  where  100%  of  munitions  were 
correctly  identified. 

3.3.2  Data  requirements 

To  meet  this  requirement  we  needed  scoring  reports  from  the  IDA. 

3.3.3  Success  criteria  evaluation  and  results 

The  objective  would  be  met  if  a  sensor- specific  dig  list  placed  all  the  TOI  before  the  no-dig  point 
and  if  additional  digs  (false  positives)  were  requested  after  all  TOI  were  identified  correctly. 
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3.3.4  Results 

This  objective  was  not  met  successfully  for  all  data  sets.  The  stop-dig  thresholds,  based  on  the 
library  matching  and  Gaussian  mixture  model  were  specified  at  the  point  before  all  TOI  were 
identified.  Our  retrospective  studies  showed  that  the  missclassified  TOI  were  surrounded  by 
other  targets,  or  had  small  SNR. 

3.4  Objective:  minimize  the  number  of  anomalies  that  cannot  be  analyzed 

Some  anomalies  may  not  be  classified,  either  because  of  the  data  are  not  sufficiently 
informative — the  sensor  physically  cannot  provide  the  data  to  support  classification  for  a  given 
target  at  a  given  depth — or  because  the  data  processing  was  inadequate.  The  former  is  a  measure 
of  instrument  performance  for  all  anomalies  for  which  all  data  analysts  converge.  The  latter  is  a 
measure  of  the  quality  of  a  data  analysis,  when  a  target  diagnostic  differs  from  those  made  by 
other  analysts. 

3.4.1  Metric 

The  metric  for  this  objective  is  the  number  of  anomalies  that  cannot  be  analyzed  by  a  particular 
method,  and  the  intersection  of  all  anomaly  lists  among  all  analysts. 

3.4.2  Data  requirements 

Each  analyst  submitted  their  anomaly  list.  IDA  scored  all  lists  and  returned  a  list  of  anomalies 
that  could  not  be  analyzed  by  any  analyst  (“cannot  analyze”  or  “failed  classification”). 

3.4.3  Success  criteria  evaluation  and  results 

The  objective  was  met  if  at  least  95%  of  a  set  of  selected  anomalies  could  be  analyzed. 

3.4.4  Results 

This  objective  was  successfully  met.  All  four  data  sets  for  all  anomalies  were  analyzed.  Not  a 
single  anomaly  was  ranked  as  “cannot  analyze.” 

3.5  Objective:  correct  estimation  of  target  parameters 

The  combined  NSMS-DE  algorithm  provides  intrinsic  and  extrinsic  parameters  for  the  different 
targets.  The  intrinsic  parameters  were  used  for  classification,  while  the  extrinsic  parameters  (i.e., 
the  target  locations)  were  utilized  for  inversion  algorithms  assessment. 

3.5.1  Metric 

The  classification  results  entirely  depend  on  how  accurately  these  parameters  are  estimated. 
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3.5.2  Data  requirements 

To  achieve  this  objective  we  inverted  and  tabulated  the  intrinsic  and  extrinsic  parameters  for  all 
targets.  To  validate  extracted  extrinsic  parameters  we  needed  results  of  intrusive  investigations. 

3.5.3  Success  criteria  evaluation  and  results 

The  objective  was  met  if  the  targets’  intrinsic  parameters  varied  within  +10%. 

3.5.4  Results 

Estimated  Classification  parameters  for  large  TOIs  were  varied  within  +  10  %.  Similar  trends 
were  achieved  for  most  middle-size  TOI-s  in  cases  of  high  SNR,  but  the  trend  diverged  for  small 
SNR  cases.  These  studies  indicated  that  more  noise-tolerant  approaches  were  needed  for  live  site 
UXO  classification.  We  responded  to  this  by  extending  the  NSMS  model  to  the  ONVMS  model 
and  adapting  the  JD  technique  for  data  pre-processing. 
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4  TEST  DESIGN 

The  only  required  test  at  the  SLO  site  entailed  collecting  training  data  for  target  characterization: 
Using  a  calibration  pit,  the  data-collection  team  made  a  series  of  static  measurements  of  example 
targets  at  several  depths  and  attitudes  in  order  to  cross-check  models,  confirm  Tx  and  Rx  polarity 
for  the  sensors,  and  characterize  the  so-called  “library  targets.” 

4.1  Site  preparation 

N/A 

4.2  Demonstration  schedule 


Preparation 

Calibration 

Blind  data  set 

Post-survey 

analysis 

Tasks  and  demonstration  stages 

Aug2009 

Sep 

’09 

Oct 

’09 

Nov 

’09 

Dec 

’09 

Jan 

’10 

Feb 

’10 

1 .  Invert  all  calibration  and  training  data 

X 

2.  Invert  5x5  TEMTADS  data 

X 

3.  Invert  MM  data 

X 

4.  Generate  MM  dig  list  and  submit  to  IDA 

X 

5.  Generate  TEMTADS  dig  list  and  submit  to 
IDA 

X 

6.  Invert  BUD  data  and  generate  dig  list 

X 

7.  Conduct  retrospective  analysis  if  needed 

X 

X 

REPORTING: 

8.  Draft  demonstration  report 

X 

9.  Final  demonstration  report 

X 

Figure  15.  Gantt  chart  showing  a  detailed  schedule  of  the  activities  conducted  at  SLO. 
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5  DATA  ANALYSIS  PLAN 

We  analyzed  all  cued  data  for  the  MetalMapper,  TEMTADS  and  BUD  sensors  and  produced 
prioritized  dig  lists  for  independent  scoring. 

5.1  Extracting  target  locations 

Target  locations  were  determined  relative  to  the  sensor  coordinate  system  using  the  differential 
evolution  and  gradient  descent  search  algorithms.  Object  responses  were  modeled  with  NSMS. 
This  combined  algorithm  was  run  for  single-target  cases  and  provided  target  locations. 

5.2  Extracting  target  intrinsic  parameters 

The  NSMS  algorithm  yields  the  targets’  intrinsic  total  NSMS,  which  we  used  for  classification. 
The  total  NSMS  contains  three  moments,  Mxx(t),  Myy(t),  and  M-Jj),  along  the  primary  axes  in  the 
target’s  own  reference  frame.  These  moments  are  similar  to  simple  dipole  moment  components 
but  carry  more  information,  accounting  for  the  targets’  inherent  heterogeneities.  The  NSMS-DE 
algorithm  outputs  the  time-decay  curves  of  the  target’s  total  ONVMS  tensor  The  next 

step  is  to  determine  the  time  decay  of  the  primary  components  of  the  total  NSMS  in  the  target’s 
reference  frame.  While  this  can  be  done  by  standard  diagonalization  (i.e.,  finding 
M(ffc)  =  \(tk)T)(tk)\r(tk),  where  V(4)  contains  the  eigenvectors  of  M(4),  it  is  more  convenient  to 
perform  a  joint  diagonalization,  M(4)  =  VDf/vjV7.  where  now  the  eigenvectors  are  shared  by  all 
time  channels;  this  allows  us  to  extract  more  reliable  total  NSMS  values  and  reduce  uncertainty. 


5.3  Selection  of  intrinsic  parameters  for  classification 

Most  UXO  are  bodies  of  revolution,  and  consequently  the  two  secondary  polarizability  elements 
are  degenerate.  However,  live-site  UXO  discrimination  studies  have  repeatedly  shown  that  this 
symmetry  can  be  compromised  due  to  low  SNR,  especially  for  small  or  deep  targets.  A  good 
classification  of  object  features  can  then  be  obtained  by  using  only  the  principal  component  of 
the  total  NSMS,  Mzz(t).  Furthermore,  to  limit  the  number  of  relevant  features  for  use  in 
classification  we  extract  parameters  exclusively  from  the  main  polarizability  Mzz(t),  both  to 
represent  size  (via  Mzz(t\ ))  and  wall  thickness  (via  Mzz(t\)  /  Mzz(tn)). 

5.4  Training 

We  used  the  supplied  training  data  set  for  algorithm  training  and  optimization.  At  the  first  stage 
of  the  process  we  used  a  semi- supervised  clustering  technique  to  identify  potential  site-specific 
TOI. 

5.5  Classification 

(a)  Probability  density  functions  were  extracted  for  single  targets. 


SERDP  1572 


26 


August  2012 


Demonstration  report 


Advanced  EMI  models  for  SLO 


(b)  All  of  the  unknown  targets  were  scored  based  on  the  probability  density  functions. 

(c)  All  items  were  further  analyzed  using  library  matching,  and  all  total  NSMS  time-decay 
curves  were  inspected  visually. 

(d)  A  classification  threshold  was  selected  and  a  final  dig  list  was  produced. 

5.6  Decision  memo 

The  algorithms  used  supplied  training  data  for  building  library  of  TOI-s  and  to  perform  inversion 
and  classification  for  the  SLO  test  are  described  in  Section  2.4.  Using  the  inversion,  clustering 
and  classification  procedures  outlined  above  we  produced  a  ranked  anomaly  list  formatted  as 
specified  by  the  IDA[18]. 
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6  COST  ASSESSMENT 

This  work  conducted  under  the  SERDP#1572  project.  There  were  some  code  development  and 
adjustment  during  these  classification  studies,  which  are  excluded  from  cost  assessment.  We  did 
our  best  to  track  time  and  resources  for  each  classification  task.  A  summary  of  the  time  needed  to 
classify  each  anomaly  appears  in  Table  4.  The  quoted  values  are,  of  course,  averages:  some 
anomalies  can  be  identified  almost  instantaneously,  while  others  require  much  more  time  and 
effort. 


Table  4:  Cost  model  for  advanced  EMI  model  demonstration  at  SLO 


Cost  Category 

Description 

Cost 

Pre-processing 

Time  required  to  analyze  training  data  and  check 
data  quality,  build  the  total  NSMS  library  for  TOI-s 

60  sec/anomaly 

Parameter  extraction 

Time  required  extract  target  feature  parameters  for 
test  data 

40  sec/anomaly 

Classifier  training 

Time  required  to  optimize  classifier  using  the 
training  data  set. 

40  sec/anomaly 

Library  matching 
technique 

Time  required  for  an  expert:  to  visually  inspect  the 
inverted  total  NSMS  and  compare  them  to  a  library 
target's  total  NSMS;  to  make  classification  decision. 

120  sec/anomaly 

Classification  and 
construction  of  a 
ranked  anomaly  list 

Time  required  to  classify  anomalies  in  the  test  set 
and  construct  the  ranked  anomaly  list 

60  sec/anomaly 

Total 

320  sec/anomaly 
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7  MANAGEMENT  AND  STAFFING 

Figure  16  is  the  organization  chart  for  the  personnel  involved  in  the  demonstration.  Their 
responsibilities  are  as  follows: 

1.  Fridon  Shubitidze  -  Principal  Investigator.  Developed  and  implemented  most  of  the 
preprocessing  and  inversion  routines  used.  Classified  MetalMapper  data  and  BUD  data 
sets  using  library  matching  and  a  Gaussian  mixture  model. 

2.  Irma  Shamatava  -  Sky  Research  Geophysicist.  Participated  in  the  inversion  and 
classification  of  TEMTADS  data. 

3.  David  Karkashadze  -  A  visiting  professor  at  Dartmouth  College.  Classified  TEMTADS 
data  using  library  matching  and  semi-supervised  parameter  clustering. 


Fridon  Shubitidze,  PI: 

MM  and  BUD  data  inversion  and  classification 


Mrs.  Irma  Shamatava: 

TEMTADS  inversion  and  classification 

Dr.  David  Karkashadze,  Dartmouth 
College:  TEMTADS  data  feature- 
parameter  clustering  and  classification 

Figure  16:  Project  management  hierarchy. 
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APPENDICES 

7.1  Appendix  A:  Health  and  Safety  Plan  (HASP) 

As  this  effort  does  not  involve  field  data  collection,  no  HASP  is  required. 

7.2  Appendix  B:  Points  of  Contact 

Points  of  contact  (POCs)  involved  in  the  demonstration  and  their  contact  information  are 
presented  in  Table  5. 

Table  5:  Points  of  Contact  for  the  advanced  EMI  models’  demonstration 


POINT  OF 
CONTACT 
Name 

ORGANIZATION 

Name 

Address 

Phone 

Fax 

E-mail 

Role  in 
Project 

Dr.  Fridon 
Shubitidze 

Sky  Research,  Inc. 

Tel:  603  643  2876 

Fax:  603-643-5161 
fridon.  shubitidze  @  skvresearch.com 

PI 

Erik  Russell 

Sky  Research,  Inc. 

3  Schoolhouse  Ln, 
Etna,  NH  03750, 
USA 

Tel:  541-552-5197 

Fax:  603-643-5161 
Erik.Russell@skvresearch.com 

Project 

Coordinator 

Dr.  Herb 
Nelson 

ESTCP  Program 
Office, 

ESTCP  Office 

901  North  Stuart  St, 
Suite  303 
Arlington,  VA 
22203-1821 

Tel:  571  372-6400 

Herbert. Nelson  @  osd.mil 

ESTCP 

Munitions 

Management 

Program 

Manager 
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